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Abstract:  This  project  addresses  two  challenges  in  complex  human-machine  systems:  i)  predicting  system 
responses  that  emerge  from  the  combined  actions  of  multiple  networked  agents  and  ii)  designing  compo¬ 
nents  to  establish  desirable  system  operations  from  collective  agent  behaviors.  In  complex  human  machine 
systems,  the  “agents”  arc  comprised  of  both  programmable  components  and  human  decision  makers.  An 
ultimate  objective  is  to  enable  effectively  controlled  coordination  of  this  combination  through  the  develop¬ 
ment  of  computational  algorithms  and  human  incentives  supported  by  analysis,  simulation,  fielded  surveys, 
and  operational  performance  feedback.  Work  has  continued  on  two  complementary  directions  towards  our 
objective.  First  is  the  analysis  and  simulation  of  agent  coordination.  Second  is  understanding  the  emer¬ 
gence  of  patterns  and  interactions  among  organizational  cultural  norms,  through  the  use  of  fielded  surveys 
and  operational  performance  feedback,  specifically  in  the  area  of  organizational  safety  for  high-risk,  high- 
consequence  systems. 
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Summary  of  Accomplishments 


Our  work  has  focused  on  three  complementary  directions  towards  our  objective.  First  is  modeling,  analysis 
and  simulation  of  agent  coordination.  Second  is  understanding  the  emergence  of  patterns  and  interactions 
among  organizational  cultural  norms,  through  the  use  of  fielded  surveys  and  operational  performance  feed¬ 
back,  specifically  in  the  area  of  organizational  safety  for  high-risk,  high-consequence  systems.  Third  is 
exploring  how  this  empirical  data  exemplifies  and  challenge  the  principles  developed  in  modeling,  analysis 
and  simulation  work. 

Multiagent  coordination:  Distributed  multiagent  systems  consist  of  a  collection  of  decision  making  com¬ 
ponents  with  limited  processing  capabilities,  locally  sensed  information,  and  limited  inter-component  com¬ 
munications,  seeking  to  achieve  interdependent  (and  sometimes  global)  objectives.  This  requires  coordi¬ 
nation.  The  distributed  nature  of  information  processing,  sensing,  and  actuation  makes  these  systems  a 
significant  departure  from  traditional  centralized  decision  architectures  and  more  aligned  with  the  frame¬ 
works  of  game  theory  (i.e.,  the  study  of  interactions  between  decision  makers)  and  information  dynamics 
(i.e.,  the  study  of  flow  and  differential  interpretation  of  information  in  populations).  Of  particular  relevance 
arc: 

•  Game  theoretic  learning,  in  which  the  focus  shifts  away  from  equilibrium  solution  concepts  and  to¬ 
wards  the  dynamics  of  how  decision  makers  reach  equilibrium.  Recent  work  explores  the  role  of  game 
theoretic  learning  as  an  approach  to  design  online  learning  algorithms  for  multiagent  coordination. 

•  Distributed  optimal  agreement,  which  builds  a  general  model  of  stochastic  agreement  dynamics  in 
large,  structured  decision  spaces,  under  limited  information  and  structured  interaction  regimes. 

Specific  work  done  in  these  areas  includes: 

•  Cooperative  control  &  potential  games:  Recent  years  have  seen  significant  interest  in  multiagent  co¬ 
ordination  from  the  viewpoint  of  “cooperative  control”.  Typical  work  in  this  year  derives  steering 
algorithms  for  coordination  of  motion  of  mobile  vehicles  and  sensors.  Our  recent  work  establishes 
the  strong  connection  between  cooperative  control  and  game  theoretic  learning,  specifically  for  so- 
called  “potential”  and  “weakly  acyclic”  games.  Indeed,  several  cooperative  control  problems  such 
as  consensus  and  dynamic  sensor  coverage  can  be  formulated  in  these  settings.  Motivated  by  this 
connection,  we  build  upon  game  theoretic  concepts  to  better  accommodate  a  broader  class  of  coop¬ 
erative  control  problems.  In  particular,  we  extend  existing  learning  algorithms  to  accommodate  (i) 
restricted  action  sets  caused  by  limitations  in  agent  capabilities  and  (ii)  group  based  decision  making. 
Furthermore,  we  also  introduce  a  new  class  of  games,  called  sometimes  weakly  acyclic  games,  for 
time-varying  objective  functions  and  action  sets,  and  provide  distributed  algorithms  for  convergence 
to  an  equilibrium. 

•  Log-linear  learning:  Log-1  incar  learning  is  a  specific  algorithm  for  multiagent  coordination  motivated 
by  statistical  physics.  In  log-linear  learning,  agents  use  stochastic  decision  rules  in  which  arc  myopi¬ 
cally  optimal  with  high  probability  while  exploring  suboptimal  possibilities  with  low  probability.  The 
traditional  analysis  of  log-linear  learning  has  centered  around  explicitly  computing  probabilities  of 
the  collective  actions  of  agents.  This  analysis  relied  on  a  highly  structured  setting:  i)  players’  utility 
functions  constitute  an  exact  potential  game,  ii)  players  update  their  strategies  one  at  a  time,  which 
we  refer  to  as  asynchrony,  iii)  at  any  stage,  a  player  can  select  any  action  in  the  action  set,  which  we 
refer  to  as  completeness,  and  iv)  each  player  is  endowed  with  the  ability  to  assess  the  utility  he  would 
have  received  for  any  alternative  action  provided  that  the  actions  of  all  other  players  remain  fixed. 
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Since  the  appeal  of  log-linear  learning  is  not  solely  the  explicit  form  of  the  stationary  distribution,  we 
seek  to  address  to  what  degree  one  can  relax  the  structural  assumptions  while  maintaining  that  only 
potential  function  maximizers  arc  the  stochastically  stable  action  profiles.  In  this  work,  we  introduce 
slight  valiants  of  log-linear  learning  to  include  both  synchronous  updates  and  incomplete  action  sets. 
In  both  settings,  we  prove  that  only  potential  function  maximizers  arc  stochastically  stable.  Further¬ 
more,  we  introduce  a  “payoff-based”  version  of  log-linear  learning,  in  which  players  arc  only  aware 
of  the  utility  they  received  and  the  action  that  they  played.  In  payoff-based  log-linear  learning,  we 
also  prove  that  only  potential  maximizers  arc  stochastically  stable.  The  key  enabler  for  these  results 
is  to  employ  methods  of  “stochastic  stability”. 

•  Efficient  network  formation:  This  work  considered  the  coordination  problem  of  network  formation, 
where  agents  can  form  and  sever  unidirectional  links  and  derive  direct  and  indirect  benefits  from  these 
links.  We  formulate  and  analyze  an  evolutionary  model  in  which  each  agents  choices  depend  on  its 
own  previous  links  and  benefits,  and  link  selections  arc  subject  to  random  perturbations.  Agents  rein¬ 
force  the  establishment  of  a  link  if  it  was  beneficial  in  the  past,  and  suppress  it  otherwise.  We  illustrate 
the  flexibility  of  the  model  to  incorporate  various  design  criteria,  including  dynamic  cost  functions 
that  reflect  link  establishment  and  maintenance,  and  distance-dependent  benefit  functions.We  show 
that  the  evolutionary  process  assigns  positive  probability  to  the  emergence  of  multiple  stable  con¬ 
figurations  (i.e.,  strict  Nash  networks),  which  need  not  emerge  under  alternative  processes  such  as 
best-reply  dynamics.  We  analyze  the  specific  case  of  so-called  frictionless  benefit  flow,  and  show  that 
a  single  agent  can  reinforce  the  emergence  of  an  efficient  network  through  an  enhanced  evolutionary 
process  known  as  dynamic  reinforcement. 

•  Self  assembly:  Self  assembly  is  a  particular-  kind  of  multiagent  coordination  problem  in  which  the 
“agents”  are  individual  components/pieces  (e.g.,  atoms)  of  a  complete  assembly  (e.g.,  molecule)  and 
are  severely  constrained  in  terms  of  their  communication  and  processing  capabilities.  This  work 
presents  methods  for  distributed  self-assembly  that  utilize  simple  rule  of  thumb  control  and  commu¬ 
nication  schemes  providing  probabilistic  performance  guarantees.  This  approach  represents  a  staunch 
departure  from  existing  approaches  that  require  more  sophisticated  control  and  communication,  but 
provide  deterministic  guarantees.  In  particular-,  we  show  that  even  under  severe  communication  re¬ 
strictions,  any  assembly  described  by  an  acyclic  weighted  graph  can  be  assembled  with  agent  memory 
and  computation  complexities  that  are  linear-  in  the  number  of  nodes  contained  in  the  desired  assembly 
graph.  We  introduce  the  concept  of  stochastic  stability  to  the  self-assembly  problem  and  show  that 
stochastic  stability  of  desirable  configurations  can  be  exploited  to  provide  probabilistic  performance 
guarantees  for  the  process.  Relaxation  of  the  communication  restrictions  allows  simple  approaches 
giving  deterministic  guarantees.  We  establish  a  clear-  relationship  between  availability  of  communi¬ 
cation  and  convergence  properties.  Self-assembly  tasks  are  considered  for  the  cases  of  many  and  few 
agents  as  well  as  large  and  small  assembly  goals.  Sensitivity  of  the  presented  process  to  communica¬ 
tion  errors  as  well  as  ill-intentioned  agents  is  considered. 

•  Agreement  Dynamics:  Agreement  is  a  fundamental  class  of  coordination  problems  that  appears  ubiq¬ 
uitously  in  distributed  systems.  Using  a  mix  of  simulation  studies  and  analytical  modes  we  study  three 
issues:  which  critical  factors  impact  the  ability  to  reach  consensus  in  a  distributed  population;  how  do 
those  factors  influence  the  type  or  quality  of  solutions  reached;  and  how  do  those  factors  impact  con¬ 
vergence  time?  We  largely  study  these  issues  in  the  domain  of  evolving  communication  and  cultural 
regimes  for  multi-agent  systems.  Our  basic  findings  are  that  a)  key  factors  include  agreement  space 
size,  agreement  space  structure,  and  network-biased  information  propagation',  b)  network  topologies 
influence  convergence  time  and  quality  by  structuring  information  propagation,  and  c)  some  network 
structures  yield  performance  characteristics  with  such  high  variance  (over  multiple  runs)  as  to  be 
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effectively  unpredictable  in  time  or  quality;  networks  arc  sometimes  uncertainty  multipliers. 

•  Information  Theory  of  Networks:  This  work  studies  the  relationship  between  network  structure/topology 
and  the  ability  to  accumulate  information.  We  have  developed  the  basics  of  an  information  theoretic 
model  that  can  help  explain  why  certain  information  dynamics  processes  (say,  consensus/agreement 
processes,  opinion  dynamics  models  like  Voter  models,  load-balancing  protocols,  or  innovation- 
diffusion)  occur  differently  under  different  network  topologies.  Briefly,  local  estimates  of  global 
states  arc  critical  for  coherent  consensus  decisionmaking — for  example  when  agents  arc  trying  to 
reach  agreement  on  some  information  state  or  some  collective  action  plan,  by  locally  converging  to 
it.  Incomplete  networks  limit  propagation  of  global  information  to  incomplete  samples  at  any  step, 
(e.g.,  an  agent  only  gets  information  from  its  network  neighbors,  not  from  all  other  agents,  hence  it 
collects  only  samples  of  the  global  state).  If  these  samples  inaccurately  reflect  global  statistics,  (e.g., 
violate  mean-field  assumptions)  then  local  decisions  can  be  globally  incoherent.  Network  topology 
is  thus  a  sampling  structure,  and  agents  must  trade  off  time  (multi-step  flows  of  information)  for  ac¬ 
curacy  when  making  local  estimates  of  global  states  in  a  networked  information  flow.  In  terms  of 
dynamics,  network  states  that  arc  changing  faster  than  information  can  propagate  these  estimates  will 
be  outdated  and  decisions  will  be  bad. 

•  Distributed  constraint  agreement  (DC A).  Individual  cognitive  states  may  be  complex  systems  of  con¬ 
straints  that  need  solving  dynamically  as  situations  evolve.  If  such  “internal”  constraint  networks 
arc  underconstrained,  many  potential  solutions  exist  per  agent.  In  many  agreement  and  coordination 
problems  (e.g.,  agreeing  on  common  communication  protocols  using  complex  languages)  popula¬ 
tions  of  agents  must  align  their  individual  constraint  solutions,  using  completely  decentralized,  and 
sometimes  limited,  information.  We  have  developed  algorithms  for  DCA  and  demonstrated  them  on 
the  problems  of  “vowel  shifting.”  Our  approach  explains  how  dynamic  vowel  systems  can  maintain 
internal  consistency  (e.g.,  ability  to  accurately  differentiate  vowel  sounds  in  real  time,  which  depends 
on  individual  inter-vowel  constraints)  and  population-wide  coherence  in  practical  time. 

Mechanism  design:  The  prior  discussion  on  multiagent  coordination  deals  with  agents  that  arc  pro¬ 
grammable  components.  As  the  terminology  implies,  complex  human-machine  systems  involve  human 
decision  makers  who  cannot  be  “programmed”  to  execute  a  desired  algorithm.  Rather,  one  must  develop 
rules  that  incentivize  rational  decision  makers  to  taking  desirable  actions  that  contribute  to  collective  per¬ 
formance,  which  need  not  be  exactly  aligned  with  individual  welfare.  This  concern  has  been  investigated  in 
the  economics  literature  under  the  framework  of  “mechanism  design”.  A  “mechanism”  is  a  policy  that  maps 
distributed  agent  messages  to  centralized  policy  decisions.  Mechanism  design  seeks  to  derive  rules  that 
incentivize  distributed  agents  to  report  “truthful”  messages,  and  consequentially  result  in  effective  policy 
decisions. 

Our  work  in  mechanism  design  explored  two  directions  motivated  by  feedback  control  considerations, 
namely,  model  uncertainty  and  dynamics. 

•  Robust  mechanism  design:  The  traditional  setup  for  mechanism  design  admits  uncertainty  in  the 
models  of  agents.  However,  this  uncertainty  must  conform  to  a  specific  and  restrictive  structure. 
An  typical  example  is  in  auctions,  where  agent  valuations  of  an  auctioned  item  arc  unknown,  but 
assumed  to  conform  to  a  specified  and  known  probability  distribution.  We  show  optimal  auctions  arc 
“fragile”  in  the  following  sense.  The  slightest  deviation  from  the  assumed  probability  distribution  can 
result  in  agents  not  participating  in  the  auction.  In  other  words,  auctions  efficiency  is  discontinuous 
with  respect  to  the  assumed  setup.  We  arc  currently  exploring  methods  to  derive  alternative  “robust” 
auctions  that  trade-off  idealized  efficiency  with  robustness. 
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•  Dynamic  mechanism  design:  Traditional  mechanism  design  deals  with  a  “one  shot”  setup,  i.e.,  the 
mechanism  is  executed  only  once.  An  emerging  topic  is  so-called  “dynamic”  mechanism  design, 
in  which  the  mechanism  is  implemented  as  paid  of  an  ongoing  dynamic  process.  An  example  is  a 
continuing  flow  of  agents  that  bid  to  utilize  limited  resources.  The  mechanism  seeks  to  accommodate 
current  agents  while  hedging  for  the  presence  or  absence  of  future  agents.  The  resulting  mechanism 
then  takes  the  form  of  a  feedback  control  policy.  This  work  is  exploring  the  use  of  feedback  control 
methods,  such  as  dissipation  inequalities,  to  construct  effective  dynamic  mechanisms. 

•  Workload  model  optimization:  Despite  the  existence  of  several  automated  air  traffic  conflict  resolution 
algorithms,  there  is  a  need  for  formulations  that  account  for  air  traffic  controller  workload.  This  work 
presents  such  an  algorithm  with  controller  workload  constraints  modeled  parametrically.  To  this  end, 
we  first  develop  an  integer  programming  model  for  general  conflict  resolution,  which  emphasizes  the 
minimization  of  fuel  costs,  and  runs  in  near  real-time.  A  parametric  procedure  based  on  this  model  is 
then  developed  to  consider  controller  workload  limitations.  Two  versions  of  the  parametric  approach 
arc  described,  along  with  computational  results.  It  is  demonstrated  that  both  formulations  can  be  used 
to  capture  a  broad  range  of  possible  controller  actions. 

Simulation  testbed  for  general  information  dynamics  problems:  We  have  constructed  a  modular  Java- 
based  simulation  system  for  modeling  general  information  dynamics  problems.  The  testbed  includes  model¬ 
ing  facilities  for  agents  with  their  cognitive  architectures  and  encode  knowledge;  three  types  of  networks  in¬ 
cluding  geographic  networks  (physical  layout  relations),  communication  networks  (information  flows,  mes¬ 
sages),  and  social  networks  (ties,  affiliations,  kinship);  Worlds  populated  with  objects,  and  sensors/effectors 
for  agents;  Measurement  and  experimentation  facilities.  This  testbed  is  running  in  prototype  form  and  is 
being  used  in  many  experiments  underpinning  the  research  above. 

Simulation  model  of  information  dynamics  in  safety-critical  systems.  We  studied  the  effects  of  infor¬ 
mation  flow  and  interpretation  (mediated  by  networks,  culture,  etc.)  on  levels  of  safety  in  safety-critical 
high-confidence  systems.  We  have  a  preliminary  model  of  how  information  flow  affects  safety  at  popula¬ 
tion  levels.  We  have  explored  mechanism  design  for  safety  critical  organizational  processes  based  on  agents 
adaptively  learning  to  propagate  information  to  the  right  consumers,  and  collectively  developing  robust  com¬ 
mon  interpretive  frames  for  message  content  over  time. 
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